# ALiBi extrapolation

MPT-30B is an open-source large language model trained by MosaicML, based on a decoder-only Transformer architecture, pre-trained on 1 trillion English text and code tokens, supporting an 8k context window and efficient inference.

MPT-30B-Chat is a dialogue generation model developed by MosaicML, fine-tuned from MPT-30B on multiple high-quality dialogue datasets, supporting multi-turn conversations and long-context processing.